Method for the computer-aided learning of a recurrent neural network for modeling a dynamic system

ABSTRACT

A method for the computer-aided learning of a recurrent neural network for modeling a dynamic system which is characterized at respective times by an observable vector with one or more observables as entries is provided. The neural network includes both a causal network with a flow of information that is directed forwards in time and a retro-causal network with a flow of information which is directed backwards in time. The states of the dynamic system are characterized by first state vectors in the causal network and by second state vectors in the retro-causal network, wherein the state vectors each contain observables for the dynamic system and also hidden states of the dynamic system. Both networks are linked to one another by a combination of the observables from the relevant first and second state vectors and are learned on the basis of training date including known observables vectors.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is the US National Stage of International ApplicationNo. PCT/EP2011/055664 filed Apr. 12, 2011, and claims the benefitthereof. The International Application claims the benefits of GermanApplication No. 10 2010 014 906.3 DE filed Apr. 14, 2010. All of theapplications are incorporated by reference herein in their entirety.

FIELD OF INVENTION

The invention relates to a method for computer-aided learning of arecurrent neural network for modeling a dynamic system and to a methodfor predicting the observables of a dynamic system on the basis of alearned recurrent neural network, and to a corresponding computerprogram product.

BACKGROUND OF INVENTION

Recurrent neural networks are used nowadays in various fields ofapplication as an appropriate way of modeling the changes over time of adynamic system such that a recurrent neural network learned usingtraining data of the dynamic system can accurately predict theobservables (observable states) of the system in question. Saidrecurrent neural network is also used to model, as states of the dynamicsystem, not only the observables but also unknown hidden states of thedynamic system, wherein generally only a causal information flow, i.e.proceeding forward in time, between consecutive states is considered.However, dynamic systems are often based on the principle that futurepredictions concerning observables also play a role in the changes overtime of the states of the system. Such dynamic systems are often onlyinadequately described by known recurrent neural networks.

SUMMARY OF INVENTION

An object is to create a method for computer-aided learning of arecurrent neural network that will provide better modeling of dynamicsystems.

This object is achieved by the independent claims. Developments of theinvention are defined in the dependent claims.

The method according to the invention is used for computer-aidedlearning of a recurrent neural network for modeling a dynamic systemwhich is characterized at respective points in time by an observablevector comprising one or more observables (i.e. observable states of thedynamic system) as entries. This method can be applied to any dynamicsystems. It can be used, for example, to model energy price and/orcommodity price movements. The method likewise enables any technicalsystem that changes dynamically over time to be modeled on the basis ofcorresponding observable state variables of the technical system inorder thereby to predict observables of the technical system using anappropriately learned network. For example, the method can be usefullyemployed to model a gas turbine and/or a wind turbine.

The recurrent neural network in the method according to the inventioncomprises a first subnetwork in the form of a causal network whichdescribes an information flow proceeding forward in time between firststate vectors of the dynamic system, wherein a first state vector at arespective point in time comprises one or more first entries which areeach assigned to an entry of the observable vector, as well a one ormore hidden (i.e. unobservable) states of the dynamic system. In orderalso to take future changes over time of the dynamic system into accountin the recurrent neural network, a second subnetwork in the form of aretro-causal network is provided, wherein the retro-causal networkdescribes an information flow proceeding backward in time between secondstate vectors of the dynamic system, wherein a second state vector at arespective point in time comprises one or more second entries which areeach assigned to an entry of the observable vector, as well as one ormore hidden states of the dynamic system. In the recurrent neuralnetwork, the observable vector at a respective point in time isdetermined such that the first entries of the first state vector arecombined with the second entries of the second state vector. Finally,the causal and the retro-causal network are learned based on trainingdata containing a sequence of consecutive known observable vectors.

The method according to the invention is characterized in that a dynamicsystem is described by a recurrent neural network which takes intoaccount both an information flow from the past to the future and aninformation flow from the future to the past. This enables dynamicsystems to be suitably modeled in which the observables at a respectivepoint in time are also influenced by predicted future observable values.

In a particularly preferred embodiment, during learning of the causaland retro-causal network at a respective point in time for which a knownobservable vector from the training data exists, the first and secondentries of the first and second state vectors are corrected using thedifference between the observable vector determined in the recurrentneural network and the known observable vector at the respective pointin time. The first and second state vectors with the corrected first andsecond entries then continue to be used for learning. In this way, at arespective point in time so-called teacher forcing is achieved wherebyobservables determined in the recurrent neural network are alwaysmatched to observables according to the training data.

In another particularly preferred embodiment, the causal andretro-causal networks are learned based on error-back-propagation withshared weights. This method of error-back-propagation with sharedweights will be sufficiently familiar to the average person skilled inthe art and is frequently used for learning in recurrent neuralnetworks. By using this method, simple and efficient learning of therecurrent neural network is achieved.

In another preferred embodiment of the method according to theinvention, in the recurrent neural network the observable vector isdetermined at a respective point in time such that the respective firstand second entries which are assigned to the same entry of theobservable vector are added.

In another embodiment of the method according to the invention, duringlearning of the causal and retro-causal network a target value isdetermined at a respective point in time for which a known observablevector according to the training data exists, which target valueconstitutes the difference vector between the observable vectordetermined in the recurrent neural network and the known observablevector at the respective point in time. Predefined here as the learningoptimization target is the minimization of the sum of the absolutevalues or squared absolute values of the difference vectors at therespective points in time for which a known observable vector from thetraining data exists. This provides a simple means of ensuring that therecurrent neural network correctly models the dynamics of the system inquestion.

In another embodiment of the method according to the invention, in thecausal network a first state vector at a respective point in time isconverted to a first state vector at a subsequent point in time bymultiplication by a matrix assigned to the causal network andapplication of an activation function. In a particularly preferredvariant, first the activation function is applied to the state vector atthe respective point in time and only subsequently is multiplication bythe matrix assigned to the causal network performed. This ensures thatobservables can be described which are not limited by the value range ofthe activation function.

In another embodiment of the method according to the invention, in theretro-causal network a second state vector at a respective point in timeis converted into a second state vector at a previous point in time bymultiplication by a matrix assigned to the retro-causal network andapplication of an activation function. Once again, first the activationfunction is preferably applied to the second state vector at therespective point in time and only subsequently is multiplication by thematrix assigned to the retro-causal network performed. This ensures alsofor the retro-causal network that observables can be described which arenot limited by the value range of the activation function.

In a particularly preferred variant, the above described activationfunctions are tanh (hyperbolic tangent) functions which are frequentlyused in recurrent neural networks.

In addition to the method described above, the invention comprises amethod for predicting observables of a dynamic system whereby theprediction is carried out using a recurrent neural network which islearned using the inventive learning process based on training datacomprising known observable vectors of the dynamic system.

The invention additionally relates to a computer program product havingprogram code stored on a machine-readable medium for carrying out themethods described above when the program is run on a computer.

BRIEF DESCRIPTION OF THE DRAWINGS

Exemplary embodiments of the invention will now be described in detailwith reference to the accompanying drawings in which:

FIG. 1 and FIG. 2 show two variants of known recurrent neural networksfor modeling a dynamic system;

FIG. 3 shows a variant of a recurrent neural network based on FIG. 2which is inventively used as a causal subnetwork;

FIG. 4 shows a variant known from the prior art for learning the causalnetwork according to FIG. 3;

FIG. 5 and FIG. 6 show variants of the learning of the causal networkfrom FIG. 3 which are used in embodiments of the method according to theinvention;

FIG. 7 shows a retro-causal network which is used in the methodaccording to the invention in combination with the causal network fromFIG. 3;

FIG. 8 and FIG. 9 show variants of the learning of the retro-causalnetwork from FIG. 7 which are used in embodiments of the methodaccording to the invention;

FIG. 10 shows an embodiment of an inventive recurrent neural networkwhich combines the networks from FIG. 3 and FIG. 7 with one another; and

FIG. 11 and FIG. 12 show embodiments of the inventive learning of therecurrent neural network illustrated in FIG. 10.

DETAILED DESCRIPTION OF INVENTION

Recurrent neural networks for modeling the behavior over time of adynamic system are sufficiently known from the prior art. These networksgenerally comprise a plurality of layers which generally contain aplurality of neurons and can be suitably learned based on training datafrom known states of the dynamic system such that future states of thedynamic system can be predicted.

FIG. 1 shows a known prior art variant of a neural network which modelsan open dynamic system. This network comprises an input layer I havingconsecutive state vectors u_(t−3), u_(t−2), u_(t−1) and u_(t) whichrepresent corresponding input variables of the dynamic system. Theseinput variables can be, for example, manipulated variables of atechnical system modeled using the neural network. The individual statevectors of the input layer I are connected to corresponding hidden statevectors s_(t−2), s_(t−1), etc. of a hidden layer via matrices B. Thehidden state vectors comprise a plurality of hidden states of thedynamic system and constitute the (unobservable) state space of thedynamic system. The individual hidden state vectors are interconnectedvia matrices A. The network additionally comprises an output layer 0having output variables in the form of state vectors y_(t−2), y_(t−1), .. . , y_(t+4) which are linked to corresponding hidden state vectorss_(t−2), s_(t−1), . . . , s_(t+4) via the matrix C. The states of theoutput layer are states of the dynamic system which result from thecorresponding input variables of the input layer I. Based on trainingdata which consists of known input variables and resulting known outputvariables, the neural network in FIG. 1 can be suitably learned usingknown methods such as error-back-propagation and then used to predictfuture output variables y_(t+1), y_(t+2), etc. in the output layer 0 onthe basis of past input variables u_(t−3), u_(t−2), . . . , u_(t−t) andthe present input variable u_(t) in the input layer I. The network inFIG. 1 is based on modeling of the dynamic system in question in theform of a superposition of an autonomous and of an externally drivensubsystem.

FIG. 2 shows another variant of a recurrent neural network which is usedin the embodiments described below of the method according to theinvention. This network models a closed dynamic system and differs fromthe network in FIG. 1 in that a distinction is no longer made betweeninput variables u_(τ) and output variables y_(τ), where τ hereinafterdenotes any point in time. Rather, both the input variables and theoutput variables are considered as observables, i.e. observable statesof an observable vector of the dynamic system. The network in FIG. 2comprises a first layer L1 and a second layer L2, wherein the firstlayer L1 represents an information flow proceeding forward in timebetween individual state vectors s_(t−2), s_(t−1), . . . , s_(t+3) ofthe modeled dynamic system. In contrast to FIG. 1, in the embodiment inFIG. 2 a state vector s_(τ) initially contains as entries the observableobservables corresponding to the state vectors y_(t) and u_(t) in FIG.1, and then the unobservable hidden states, wherein the number of hiddenstates is generally much greater than the number of observables. Theindividual state vectors in the layer L1 are converted into one anotherby matrices A which are suitably learned based on training data. At thestart of said learning, in the layer L1 a suitable bias is defined whichis denoted by S₀ in FIG. 2 and also in all the subsequent figures.

A suitably learned recurrent neural network as shown in FIG. 2 suppliesin the second layer the observables y_(t−1), u_(t−2), y_(t−1), u_(t−1),. . . , etc. at the respective points in time. The entries of thecorresponding state vectors s_(τ), which entries correspond toobservables, are obtained via the matrix [Id, 0]. For the columns, thematrix [Id, 0] has the dimension of the state vector s_(τ) and, for therows, the dimension according to the number of observables. Theleft-hand part of the matrix forms a square identity matrix and, for theremaining columns, the matrix contains only zeros by means of which thefiltering of the observables from the state vector s_(τ) is achieved.With the network in FIG. 2, the observables are embedded in a largestate vector s_(τ), thereby achieving dynamically consistent dynamicsystem modeling that is symmetrical in all the variables, wherein timeplays no specific role. The network in FIG. 2 also represents a causalnetwork, as the information flow between the states of the layer L1progresses forward in time from the past to the future.

FIG. 3 shows a recurrent neural network based on FIG. 2, wherein now allthe observables are consistently denoted as observable vectors y_(t−6),y_(t−5), . . . , y_(t+3). The notation y_(τ) therefore comprises boththe output variable y_(τ) and the input variable u_(τ) from FIG. 2. Thisnotation will also be used in the following for all the other recurrentneural network variants described. In addition, in FIG. 3 the observablevectors y_(t+1), y_(t+2), and y_(t+3) to be predicted using the networkare indicated by dashed circles for the sake of clarity, i.e. thepresent point in time is denoted by t in FIG. 3 and also in all theother figures. Past points in time are therefore the time instants t−1,t−2, etc. and future points in time are the time instants t+1, t+2, t+3,etc.

FIG. 4 shows a known variant of the learning of the recurrent neuralnetwork in FIG. 3, where y^(d) _(t−3), y^(d) _(t−2), y^(d) _(t−1) andy^(d) _(t) represent known observable vectors according to predefinedtraining data of the dynamic system to be modeled. The matrix [Id, 0]corresponds to the above explained matrix for filtering the observablesfrom the corresponding state vector s_(τ). On the other hand, the matrix

$\quad\begin{bmatrix}{Id} \\0\end{bmatrix}$

enables the known observable vector y^(d) _(t) to be converted into anobservable vector which contains not only the entries for the knownobservables but also entries for the other hidden states which, however,are all set to zero. This matrix

$\quad\begin{bmatrix}{Id} \\0\end{bmatrix}$

comprises a number of columns corresponding to the number of observablesand a number of rows corresponding to the dimension of the state vectors_(τ). In the upper portion, the matrix forms a square identity matrixand the remaining rows of the matrix contain exclusively zeros. Thenetwork in FIG. 4 additionally contains the matrix C with which a states_(τ) is transitioned to a state r_(τ). Said state r_(τ) represents afiltered state which contains only the hidden states of the vectors_(τ). Consequently, the matrix C is a matrix which contains ones on thediagonal elements corresponding to the corresponding rows or columns ofthe hidden states and whose remaining entries are set to zero.

The linking shown in FIG. 4 of the known states y^(d) _(τ) with thestate r_(τ) ensures that the observable values obtained by the neuralnetwork are replaced by the observables y^(d) _(τ) according to thetraining data. Replacement of the determined observables by the actualobservables according to the training data is therefore achieved in eachtime step τ≦t. Such a learning method is also known as “teacherforcing”. According to the representation in FIG. 4, the followingrelationships are modeled using the recurrent neural network, wherein—asmentioned above—the time t corresponds to the current present time:

$\begin{matrix}{{\tau \leq {t\text{:}\mspace{14mu} s_{\tau + 1}}} = {\tanh\left( {A\left( \underset{\underset{r_{\tau}}{}}{{\begin{bmatrix}0 & 0 \\0 & {Id}\end{bmatrix}s_{\tau}} + {\begin{bmatrix}{Id} \\0\end{bmatrix}y_{\tau}^{d}}} \right)} \right)}} & (1) \\{{\tau > {t\text{:}\mspace{14mu} s_{\tau + 1}}} = {\tanh \left( {As}_{\tau} \right)}} & (2) \\{{{for}\mspace{14mu} {all}\mspace{14mu} \tau \text{:}\mspace{14mu} y_{\tau}} = {\left\lbrack {{Id},0} \right\rbrack {s_{\tau}.}}} & (3)\end{matrix}$

The learning is based on the following optimization target:

$\begin{matrix}\left. {\sum\limits_{\tau = {t - m}}^{t}\; \left( {y_{\tau} - y_{\tau}^{d}} \right)^{2}}\rightarrow{\min\limits_{A}.} \right. & (4)\end{matrix}$

In other words, the matrix A is sought which minimizes the quadraticerror, summed over the time instants t-m≦τ≦t, between observable vectorsdetermined via the network and known observable vectors.

The teacher forcing described above is also employed in the recurrentneural network used in the method according to the invention, but inmodified variants which are illustrated in FIGS. 5 and 6 for the causalnetwork in FIG. 3. Similar notations to FIG. 4 are retained (except forany signs). The additional matrix Id in FIG. 5 denotes a correspondingidentity mapping for the state vector at which the arrow denoted by thematrix begins. In contrast to the embodiment in FIG. 4, a targetvariable or target value tar is now introduced in FIG. 5 whichrepresents the difference vector between the observable vector y_(τ)determined by the recurrent neural network within the state vector s_(τ)and the known observable vector y^(d) _(τ). This target value, which isideally zero, is in turn used to replace the corresponding determinedobservables in the vectors s_(τ) by the known observables according tothe training data, which is expressed by the linking via the matrix

$\begin{bmatrix}{- {Id}} \\0\end{bmatrix}.$

Using the structure of the network according to FIG. 5, the followingequations are modeled:

$\begin{matrix}{{\tau \leq {t\text{:}\mspace{14mu} s_{\tau + 1}}} = {\tanh\left( {A\left( \underset{\underset{r_{\tau}}{}}{s_{\tau} - {\begin{bmatrix}{Id} \\0\end{bmatrix}\left( {y_{\tau} - y_{\tau}^{d}} \right)}} \right)} \right)}} & (5) \\{{\tau > {t\text{:}\mspace{14mu} s_{\tau + 1}}} = {\tanh \left( {As}_{\tau} \right)}} & (6) \\{{{for}\mspace{14mu} {all}\mspace{14mu} \tau \text{:}\mspace{14mu} y_{\tau}} = {\left\lbrack {{Id},0} \right\rbrack {s_{\tau}.}}} & (7)\end{matrix}$

Similarly to the network in FIG. 4, the optimization target is given by:

$\begin{matrix}\left. {\sum\limits_{\tau = {t - m}}^{t}\; \left( {y_{\tau} - \gamma_{\tau}^{d}} \right)^{2}}\rightarrow{\min\limits_{A}.} \right. & (8)\end{matrix}$

Using the architecture according to FIG. 5, because of the tanh functionused, only observables between −1 and +1 can be modeled, as matrixmultiplication by the matrix A is performed first and only then is thetanh function applied which has a value range between −1 and 1. In amodified variant of the learning according to FIG. 5, the tanh functionis applied to the corresponding state r_(τ) or s_(τ) first, and onlythen is matrix multiplication by the matrix A performed. Such a variantof the network is illustrated in FIG. 6, wherein the application of thetanh function before matrix multiplication by the matrix A isillustrated in that the tanh function is now depicted in the circleswhich in FIG. 5 contain the states r_(τ), and also between the statess_(t+1) and s_(t+2). According to this variant, observables outside thevalue range between −1 and +1 can also be modeled. FIG. 6 shows apreferred learning variant which is also used in the inventive neuralnetwork structure described below. The difference between the recurrentneural network in FIG. 6 and the recurrent neural network in FIG. 5 canbe expressed mathematically in that, in the above equations (5) and (6),the position of the matrix A is transposed with the position of thefunction tanh.

In the preceding, suitable learning of a causal network having aninformation flow proceeding forward in time was described. The inventionis based on the insight that a causal modal is not always suitable fordescribing a dynamic system. In particular, there are dynamic systemswhich also have a retrocausal information flow in the reverse timedirection from the future to the present. These are dynamic systemswhose changes over time are influenced by planning involving theprediction of future observables. For the change over time of acorresponding state vector of the dynamic system, not only precedingstate vectors but also predicted future state vectors are thereforetaken into account. For example, regarding the market price movements ofenergy or commodities, the price is determined not only by supply anddemand, but also by planning aspects of the sellers/buyers for thesale/purchase of energy or commodities.

The method according to the invention is based on the concept ofmodeling a dynamic system such that an information flow is considerednot only in the causal direction from the past to the future, but alsoan information flow in the retro-causal direction from the future to thepast. Such an information flow can be implemented by a retro-causalnetwork. Such a network is depicted in FIG. 7. The network in FIG. 7differs from the network in FIG. 3 in that the information flow betweenthe states s_(τ) runs in reverse direction from the future to the past,the process being again initialized using a bias S₀ which now, however,is a state in the future. Analogously to the network in FIG. 3, thenetwork in FIG. 7 can be learned via the minimization of a target valuetar, as indicated in FIG. 8. FIG. 8 corresponds to the representation inFIG. 5 except that the causality direction is now reversed. Theequations (5) to (8) can be applied analogously, with the differencethat s_(τ+1) is replaced by s_(τ−1) in equations (5) and (6). The abovedescribed teacher forcing for learning the network can therefore also beused for the retro-causal network. Likewise, the learning shown in FIG.6, in which first the tanh function and only then matrix multiplicationis applied at the transition from one state to a successor state, canalso be used analogously for the retro-causal network. This isillustrated in FIG. 9 which corresponds to the representation in FIG. 6,with the difference that the information flow proceeds from the futureto the present.

The invention is henceforward based on a combination of a causal networkwith a retro-causal network, thereby providing a recurrent neuralnetwork having an information flow both from the past to the future andfrom the future to the past. This makes it possible to also modeldynamic systems in which predicted future states also play a role in thedynamic progression of the states.

FIG. 10 shows in generic form an inventive combination of a causalnetwork with a retro-causal network, thereby creating a recurrent neuralnetwork which can be learned in a suitable manner The lower part of thisnetwork is composed of a causal network N1 and the upper part iscomposed of a retro-causal network N2. The network N1 corresponds to thecausal network in FIG. 3 and the network N2 corresponds to theretro-causal network in FIG. 7, wherein in the retro-causal network thematrices are now denoted by A′ and the states by s_(τ)′, since matricesand states for the causal and the retro-causal network can be different.The two networks are interlinked by the corresponding observable vectory_(τ).

Based on the network in FIG. 10, FIG. 11 shows the learning of thenetwork by means of teacher forcing. In the preceding, said teacherforcing has been explained separately for the causal network in FIG. 6and the retro-causal network in FIG. 9. In FIG. 11, for example, theobservables contained in the state vector s_(t) are denoted by Δ_(t) andthe observables contained in the state vector s_(t)′ by Δ_(t)′ for thetime t. The sum of Δ_(t) and Δ_(t)′ represents the observable vectordetermined by the recurrent network and the target value is thedifference between this sum and the actual observable vector y^(d) _(t)according to the training data. By linking the target values via thecorresponding matrices

$\quad\begin{bmatrix}{- {Id}} \\0\end{bmatrix}$

to the state vector s_(τ) or s_(τ)′, teacher forcing is again achievedfor each time step τ≦t. In FIG. 11 the corresponding state r_(τ) orr_(τ)′ resulting from teacher forcing is specified e.g. only for thetime instant τ32 t. To this state is then applied first the tanhfunction and then multiplication by the matrix A or A′.

In order to implement learning according to FIG. 11,error-back-propagation with shared weights, a method sufficiently knownfrom the prior art, is used as shown in FIG. 12. Error-back-propagationwith shared weights is achieved in that error-back-propagation iscalculated once for the causal network N1 and once for retro-causalnetwork N2 in two copies of the network in FIG. 11, it beingsimultaneously ensured that the same matrix A is always used in bothcopies of the network and the same matrix A′ is always used in bothcopies of the network. Error-back-propagation with shared weights issufficiently well known to the average person skilled in the art andwill not therefore be explained in further detail.

The inventive method described in the foregoing has a number ofadvantages. In particular, dynamic systems can also be learned in whichfuture predicted states of the dynamic system influence the currentstate. The method can be used for different dynamic systems. Forexample, the dynamic system can represent the changes over time ofenergy or more specifically electricity prices and/or commodity prices,wherein various types of energy (e.g. gas, oil) and/or commodities aswell as other economic factors such as the conversion of differentcurrencies and share indices can be taken into account as observables.Using a recurrent neural network learned by appropriate training data,suitable predictions concerning future price movements for energy and/orcommodities can be made. Another field of application is modeling thedynamic behavior of a technical system. For example, the recurrentneural network according to the invention can be used to predict theobservable states of a gas turbine and/or of a wind turbine or also ofany other technical systems.

1.-15. (canceled)
 16. A method for computer-aided learning of arecurrent neural network for modeling a dynamic system which ischaracterized at respective times by an observable vector comprising oneor more observables as entries, the method comprising: providing arecurrent neural network comprising a causal network, a retro-causalnetwork, an observable vector with one or more observables, wherein thecausal network describes an information flow proceeding forward in timebetween first state vectors of the dynamic system, wherein a first statevector at a respective point in time comprises one or more first entrieswhich are each assigned to an entry of the observable vector, and one ormore hidden states of the dynamic system, wherein the retro-causalnetwork describes an information flow proceeding backward in timebetween second state vectors of the dynamic system, wherein a secondstate vector at a respective point in time comprises one or more secondentries which are each assigned to an entry of the observable vector,and one or more hidden states of the dynamic system, determining theobservable vector by combining the first entries of the first statevector with the second entries of the second state vector, wherein thecausal network and the retro-causal network are learned based ontraining data which contains a sequence of consecutive known observablevectors.
 17. The method as claimed in claim 16, wherein, during learningof the causal and retro-causal networks at a respective point in time,for which a known observable vector from the training data exists, thefirst and second entries of the first and second state vectors arecorrected using the difference between the observable vector determinedin the recurrent neural network and the known observable vector at therespective point in time, the first and second state vectors with thecorrected first and second entries continuing to be used for learning.18. The method as claimed in claim 16, wherein the causal network andthe retro-causal network are learned based on error-back-propagationwith shared weights.
 19. The method as claimed in claim 16, wherein, inthe recurrent neural network at a respective point in time, theobservable vector is determined such that the respective first andsecond entries which are assigned to the same entry of the observablevector are added.
 20. The method as claimed in claim 16, wherein, duringlearning of the causal and retro-causal networks at a respective pointin time, for which a known observable vector from the training dataexists, a target value is determined which represents the differencevector between the observable vector determined in the recurrent neuralnetwork and the known observable vector at the respective point in time,wherein the minimization of the sum of the absolute values or squaredabsolute values of the difference vectors at the respective points intime, for which a known observable vector from the training data exists,is predefined as the learning optimization target.
 21. The method asclaimed in claim 16, wherein in the causal network a first state vectorat a respective point in time is converted into a first state vector ata subsequent point in time by multiplication by a matrix assigned to thecausal network and the application of an activation function.
 22. Themethod as claimed in claim 21, wherein first the activation function isapplied to the first state vector at the respective point in time andthen multiplication by the matrix assigned to the causal network isperformed.
 23. The method as claimed in claim 16, wherein in theretro-causal network a second state vector at a respective point in timeis converted into a second state vector at a previous point in time bymultiplication by a matrix assigned to the retro-causal network and theapplication of an activation function.
 24. The method as claimed inclaim 23, wherein first the activation function is applied to the secondstate vector at the respective point in time and then multiplication bythe matrix assigned to the retro-causal network is performed.
 25. Themethod as claimed in claim 21, wherein the activation function is a tanhfunction.
 26. The method as claimed in claim 16, wherein the recurrentneural network is used to model energy price and/or commodity pricechanges over time.
 27. The method as claimed in claim 16, wherein therecurrent neural network is used to model a technical system.
 28. Themethod as claimed in claim 27, wherein the technical system is a gasturbine or a wind turbine.
 29. A non-transitory computer readable mediumcomprising program code for carrying out a method when the program isexecuted on a computer, wherein the method is for computer-aidedlearning of a recurrent neural network for modeling a dynamic systemwhich is characterized at respective times by an observable vectorcomprising one or more observables as entries, the method comprising:using a recurrent neural network comprising a causal network, aretro-causal network, an observable vector with one or more observables,wherein the causal network describes an information flow proceedingforward in time between first state vectors of the dynamic system,wherein a first state vector at a respective point in time comprises oneor more first entries which are each assigned to an entry of theobservable vector, and one or more hidden states of the dynamic system,wherein the retro-causal network describes an information flowproceeding backward in time between second state vectors of the dynamicsystem, wherein a second state vector at a respective point in timecomprises one or more second entries which are each assigned to an entryof the observable vector, and one or more hidden states of the dynamicsystem, determining the observable vector by combining the first entriesof the first state vector with the second entries of the second statevector, wherein the causal network and the retro-causal network arelearned based on training data which contains a sequence of consecutiveknown observable vectors.